📌 Detecting Floods for Disaster Relief
How good are you at detecting floods?
- Sort data.
- Load data
- Instantiate and Train Model
- Visualizing Mistakes
- Model Inference
- Improving the model.
- Takeaways
You can find this notebook on Kaggle here.
The model that will be created in this notebook can detect whether an area shown in an image is flooded or not. The idea for creating this model has been spurred from the recent floodings in Pakistan.
Such models can prove useful in flood relief, helping to detect which areas need immediate focus.
The dataset used to train this model is Louisiana flood 2016, uploaded by Kaggle user Rahul T P, which you can view here.
The fastai library, a high level PyTorch library, has been used.
One of the points of this notebook is to showcase how simple it is to create powerful models. That said, this notebook is not a tutorial or guide.
from fastai.vision.all import *
The data in the dataset needs to be organized into train and valid folders. Each will contain the same subfolders, 0 and 1, which will be used to label the data. A label of 0 indicates the area shown in the image is not flooded, while a label of 1 indicates the area shown in the image is flooded.
The images in the dataset itself has been organized as follows:
    If no underscore is in the file name, the image shows the area before or after the flood.
    If an underscore is in the file name, the image shows the area during the flood:
- If a zero follows the underscore, the area was not flooded.
- If a one follows the underscore, the area was flooded.
Creating the necessary paths.
working_path = Path.cwd(); print(working_path)
folders = ('train', 'valid')
labels = ('0', '1')
input_path = Path('/kaggle/input')
train_image_paths = sorted(input_path.rglob('train/*.png'))
valid_image_paths = sorted(input_path.rglob('test/*.png'))
len(train_image_paths), len(valid_image_paths)
Creating the necessary directories.
for folder in folders:
if not (working_path/folder).exists():
(working_path/folder).mkdir()
for label in labels:
if not (working_path/folder/label).exists():
(working_path/folder/label).mkdir()
Move images to new directories.
try:
for image_path in train_image_paths:
if '_1' in image_path.stem:
with (working_path/'train'/'1'/image_path.name).open(mode='xb') as f:
f.write(image_path.read_bytes())
else:
with (working_path/'train'/'0'/image_path.name).open(mode='xb') as f:
f.write(image_path.read_bytes())
except FileExistsError:
print("Training images have already been moved.")
else:
print("Training images moved.")
try:
for image_path in valid_image_paths:
if '_1' in image_path.stem:
with (working_path/'valid'/'1'/image_path.name).open(mode='xb') as f:
f.write(image_path.read_bytes())
else:
with (working_path/'valid'/'0'/image_path.name).open(mode='xb') as f:
f.write(image_path.read_bytes())
except FileExistsError:
print("Testing images have already been moved.")
else:
print("Testing images moved.")
Check that images have been moved.
training_images = get_image_files(working_path/'train'); print(len(training_images))
Image.open(training_images[0])
validation_images = get_image_files(working_path/'valid'); print(len(validation_images))
Image.open(validation_images[-1])
Create the training and validation dataloaders through fastai's quick and easy DataBlock class.
dataloaders = DataBlock(
blocks = (ImageBlock, CategoryBlock),
get_items = get_image_files,
splitter = GrandparentSplitter(),
get_y = parent_label,
item_tfms = [Resize(192, method='squish')]
).dataloaders(working_path, bs=32)
Check that data has been loaded correctly.
dataloaders.show_batch(max_n=8)
learner = vision_learner(dataloaders, resnet18, metrics=error_rate)
learner.fine_tune(9)
Nice! A relatively low error rate for no tweaking.
We have to see how the model is getting confuzzled.
interp = ClassificationInterpretation.from_learner(learner)
interp.plot_confusion_matrix()
Only a couple of mistakes. Let's see what they are.
interp.plot_top_losses(5, nrows=1)
Nothing has been mislabeled, but the first one is especially tricky to determine, even for human eyes.
Let's test the model on some images of the recent flooding in Pakistan.
def infer_image(image_path):
display(Image.open(image_path))
label, _, probabilities = learner.predict(PILImage(PILImage.create(image_path)))
if label == '0':
print(f"The area shown in the image is not flooded with probability {probabilities[0]*100:.2f}%.")
elif label == '1':
print(f"The area shown in the image is flooded with probability {probabilities[1]*100:.2f}%.")
else:
print("Unknown label assigned to image.")
infer_image(input_path/'floodclassifiertestset'/'1'/'1.jpeg')
Not bad!
Let's try it on another image.
infer_image(input_path/'floodclassifiertestset'/'1'/'2.jpg')